Apparatus and method for generating a multi-viewpoint image

ABSTRACT

According to one embodiment, in an apparatus for generating a multi-viewpoint image, a separation unit separates a target image into a first diffuse reflection image and a first non-diffuse reflection image based on a pixel value of each pixel of the target image. The first non-diffuse reflection image has components except for the first diffuse reflection image. A first estimation unit estimates a change amount of each pixel among a plurality of first non-diffuse reflection images corresponding to viewpoints differently. A first generation unit generates a second non-diffuse reflection image by changing at least one of a shape and a luminance of each pixel of the first non-diffuse reflection image, based on the change amount of each pixel. A synthesis unit generates the multi-viewpoint image by synthesizing the first diffuse reflection image with the second non-diffuse reflection image. Each viewpoint image corresponds to each of the viewpoints.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2012-220828, filed on Oct. 2, 2012; theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an apparatus and amethod for generating a multi-viewpoint image from at least one targetimage.

BACKGROUND

It is known that a human perceives brightness of the metal andsmoothness of the surface of the water from a difference of intensitiesbetween reflected lights incident onto the right and left eyes, or asubtle difference between directions of the reflected lights.Conventional technique by using this characteristic is disclosed. Here,by detecting a region speculatively reflected in an image, a positionand a size of the region are changed between an image for the right eyeand an image for the left eye.

However, in this conventional technique, when the position and the sizeof the region speculatively reflected are changed, a distortion or abreak often occurs on an image newly generated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an image processing apparatus according tothe first embodiment.

FIG. 2 is a flow chart of processing of the image processing apparatusaccording to the first embodiment.

FIG. 3 is a block diagram of the image processing apparatus according tothe second embodiment.

FIG. 4 is a flow chart of processing of the image processing apparatusaccording to the second embodiment.

FIG. 5 is a schematic diagram to explain a method for estimating aparallax vector.

FIG. 6 is a block diagram of the image processing apparatus according tothe third embodiment.

FIG. 7 is a flow chart of processing of the image processing apparatusaccording to the third embodiment.

FIG. 8 is a schematic diagram to explain all operation of the imageprocessing apparatus according to the third embodiment.

FIG. 9 is a schematic diagram to explain a relationship between aviewpoint position and the image according to the third embodiment.

FIG. 10 is a schematic diagram to explain a method forestimating/extracting a change amount of gloss between viewpointsaccording to the third embodiment.

FIG. 11 is a schematic diagram to explain a method for generating amulti-viewpoint image using the change amount according to the thirdembodiment.

DETAILED DESCRIPTION

According to one embodiment, an image processing apparatus generates amulti-viewpoint image from at least one target image. The imageprocessing apparatus includes a separation unit, a first estimationunit, a first generation unit, and a synthesis unit. The separation unitis configured to separate the target image into a first diffusereflection image and a first non-diffuse reflection image based on apixel value of each pixel of the target image. The first non-diffusereflection image has components except for the first diffuse reflectionimage. The first estimation unit is configured to estimate a changeamount of each pixel among a plurality of first non-diffuse reflectionimages corresponding to viewpoints differently. The first generationunit is configured to generate a second non-diffuse reflection image bychanging at least one of a shape and a luminance of each pixel of thefirst non-diffuse reflection image, based on the change amount of eachpixel. The synthesis unit is configured to generate the multi-viewpointimage by synthesizing the first diffuse reflection image with the secondnon-diffuse reflection image. Each viewpoint image corresponds to eachof the viewpoints.

Various embodiments will be described hereinafter with reference to theaccompanying drawings.

(The First Embodiment)

As to the image processing apparatus of the first embodiment, an inputimage (target image) to be processed is separated into an image havingpixels due to diffuse reflection of each pixel value (Hereinafter, it iscalled “diffuse reflection image”) and an image having pixels due toother components except for the diffuse reflection (Hereinafter, it iscalled “non-diffuse reflection image”). The diffuse reflection imagerepresents image components (such as an object's color) of which colorand luminance are not changed by a viewpoint position. Furthermore, thenon-diffuse reflection image represents image components except for thediffuse reflection image. For example, the image components (such as agloss or refracted light) are color and luminance changed by theviewpoint position.

The input image includes a pixel value of each pixel. For example, thepixel value is an image having a luminance signal and a color-differencesignal based on standard of International Telecommunication Union(Hereinafter, it is called “ITU”). These signals may be based on any ofa system to have RGB (three primary colors) components and a system toconvert from RGB to the luminance signal and the color-differencesignal. In the first embodiment, as one example, the system to have RGBcomponents corresponding to three primary colors based on ITU-RBT 0.601standard. Accordingly, a pixel value of each pixel in the input image isrepresented as R-channel having a luminance of red component, G-channelhaving a luminance of green component, and B-channel having a luminanceof blue component. R-channel has discrete pixel values of 0˜r₀,G-channel has discrete pixel value of 0˜g₀, and B-channel has discretepixel values of 0˜b₀. Moreover, the input image may be a single image orsequential images.

Moreover, in the first embodiment, a method for generating two outputimages corresponding to two viewpoints (right eye, left eye) from oneinput image will be explained as an example. However, the method is notlimited to this. For example, the output image may be at least threeimages corresponding to at least three viewpoints. This case is suitablefor usage to stereoscopically view by naked eyes. Furthermore, the inputimage may be a plurality of images, and each image may mutually have aparallax in correspondence with a different viewpoint. In this case,each input image is separated into a diffuse reflection image andnon-diffuse reflection image. Furthermore, as one of the output images,the input image may be utilized as it is.

FIG. 1 is a block of the image processing apparatus of the firstembodiment. The image processing apparatus includes a separation unit100, a generation unit 200, and a synthesis unit 300.

The separation unit 100 separates the input image (having a pixel valueof each pixel) into a diffuse reflection image and non-diffusereflection image.

The generation unit 200 generates non-diffuse reflection image for righteye and non-diffuse reflection image for left eye by changing the pixelvalue of each pixel of the non-diffuse reflection image. As tonon-diffuse reflection image to be presented to the right eye andnon-diffuse reflection image to be presented to the left eye, the pixelvalue of the non-diffuse reflection image is changed by two differentmethods.

The synthesis unit 300 synthesizes the diffuse reflection image with thenon-diffuse reflection image for right eye and the non-diffusereflection image for left eye respectively, and generates an outputimage corresponding to each viewpoint. Specifically, by adding thenon-diffuse reflection image for right eye and the non-diffusereflection image for left eye to the diffuse reflection imagerespectively, two images to be presented to the right eye and the lefteye are generated.

Next, operation to generate the images according to the first embodimentis explained. FIG. 2 is a flow chart of operation of the imageprocessing apparatus of the first embodiment.

The separation unit 100 separates an input image I into a diffusereflection image and non-diffuse reflection image (S11). In the firstembodiment, a method for separating into the diffuse reflection imageand the non-diffuse reflection image based on dichromatic reflectionmodel is used. Briefly, the non-diffuse reflection image is regarded asa specular reflection image. However, the non-diffuse reflection imageis not limited to the specular reflection image. For example, separationmay be performed by using a model in which a refracted light or asubsurface scattered light can be taken into consideration.

A light incident onto an object (subject) is reflected via two physicaldifferent paths. One is reflected on a boundary of the surface of theobject, which is called “specular reflection”. The other is due toscattering of light incident onto uneven surface of the object, which iscalled “diffuse reflection”. The diffuse reflection includes a color ofthe surface of the object itself different from a color of the lightsource. Assume that reflection characteristic of the object in the inputimage I is based on dichromatic reflection model. I is represented byfollowing equation (1).I=I ^(diff) +I ^(spec)  (1)

In the equation (1), I^(diff) represents the diffuse reflection imageand I^(spec) represents the specular reflection image. Here, RGB valuesC_(R), C_(G) and C_(B) of each pixel of the image are represented byfollowing equation (2).

$\begin{matrix}{\begin{pmatrix}C_{R} \\C_{G} \\C_{B}\end{pmatrix} = {{\alpha \times \begin{pmatrix}D_{R} \\D_{G} \\D_{B}\end{pmatrix}} + {\beta \times \begin{pmatrix}S_{R} \\S_{G} \\S_{B}\end{pmatrix}}}} & (2)\end{matrix}$

In the equation (2), D_(R), D_(G) and D_(B) represent RGB values ofdiffuse reflection characteristic, S_(R), S_(G) and S_(B) represent RGBvalues of specular reflection characteristic, α represents diffusereflection coefficient, and β represents specular reflectioncoefficient. In order to calculate I^(diff) and I^(spec) from the inputimage I, “D_(R), D_(G) and D_(B) with α”, “S_(R), S_(G) and S_(B) withβ”, are respectively calculated for each pixel. Hereinafter, one exampleof the method for calculating will be explained.

First, values of S_(R), S_(G) and S_(B) are determined on assumptionthat color ratio thereof is “1:1:1”. Briefly, the illumination color isassumed as white. Next, values of D_(R), D_(G) and D_(B) are calculatedfrom a set having similar hue among pixel values of around pixels.Furthermore, by solving the equation (2) with D_(R), D_(G) and D_(B)estimated, α and β are calculated. As mentioned-above, values necessaryto calculate I^(diff) and I^(spec) are calculated, and the input imageis separated.

The generation unit 200 generates non-diffuse reflection image I′^(spec)_(L) for left eye and non-diffuse reflection image I′^(spec) _(R) forright eye, based on I^(spec) (S12). Here, difference is set betweenI′^(spec) _(L) and I′^(spec) _(R). For example, as shown in an equation(3), by subjecting I^(spec) to gamma transform with two different gammavalues, two images I′^(spec) _(L) and I′^(spec) _(R) are generated. Adifference of luminance is set between I′^(spec) _(L) and I′^(spec)_(R).I′ ^(spec) _(L)=(I ^(spec))^(−γ) ^(L)I′ ^(spec) _(R)=(I ^(spec))^(−γ) ^(R)   (3)

A method for setting the difference between I′^(spec) _(L) and I′^(spec)_(R) is not limited to above-mentioned method. For example, by changinga parameter to enlarge/reduce, the difference may be set betweenI′^(spec) _(L) and I′^(spec) _(R). Alternatively, by using non-lineartransform by morphing, the shape may be changed. In latter case,specifically, following method may be used.

By dividing a specular reflection image into each gloss region, thedifference may be set between I′spec_(L) and I′^(spec) _(R). In order todivide the specular reflection image into each gloss region, bybinarizing the specular reflection image with some threshold, labelingprocessing may be applied. Furthermore, a conventional method fordividing into regions may be used. By centering a center of gravity ofthe gloss region divided, and by enlarging or reducing the size of glossregion of I′^(spec) _(L) and I′^(spec) _(R) with differentenlargement/reduction ratio respectively, a difference of gloss can beset between two output images for right eye and left eye. Alternatively,by morphing, a part of the gloss region of I′^(spec) _(L) may beenlarged, and, conversely, a part of the gloss region of I′^(spec) _(R)may be reduced.

The synthesis unit 300 synthesizes the diffuse reflection image I^(diff)with I′^(spec) _(L) and I′^(spec) _(R), and generates an image for lefteye I′_(L) and an image for right eye I′_(R) (S13). In the firstembodiment, by assuming the dichromatic reflection model, an outputimage is generated according to an equation (4).I′ _(L) =I ^(diff) +I′ ^(spec) _(L)I′ _(R) =I ^(diff) +I′ ^(spec) _(R)  (4)

In conventional technique, a position and a size of the specularreflection region are changed without distinguishing between the diffusereflection image and non-diffuse reflection image (specular reflectionimage). Accordingly, when the size and the position of glosses arechanged, the diffuse reflection image is influenced thereby, and adistortion or a break often occurs in the image generated.

In the first embodiment, by separating an image component (in an inputimage) changeably viewed according to a person's viewpoint, signalvalues of this image component are changed. After that, this imagecomponent is synthesized with other image components in the input image.Accordingly, change executed to the non-diffuse reflection image doesnot degrade the diffuse reflection image. As a result, an image of whichquality such as brightness is improved can be generated.

(The Second Embodiment)

The second embodiment relates to an apparatus for generating at leasttwo output images (mutually having parallax) for stereoscopic visionfrom the input image.

FIG. 3 is a block diagram of an image processing apparatus according tothe second embodiment. The image processing apparatus includes theseparation unit 100, the generation unit 200, the synthesis unit 300,and a generation unit 400. The generation unit 400 generates a parallaximage based on depth information of the input image.

Next, image generation operation of the second embodiment will beexplained. FIG. 4 is a flow chart of processing of the image processingapparatus of the second embodiment. Hereinafter, the case that the inputimage is one and depth information is inputted will be explained as anexample. Furthermore, a component that the diffuse reflection image andnon-diffuse reflection image (in the input image) are separated andparallax is assigned to respective images based on the depth informationwill be explained. However, the second embodiment is not limited to thiscomponent. Input information may be a parallax vector. Furthermore,after an image to which parallax is assigned based on the depthinformation is generated, the diffuse reflection image and non-diffusereflection image (in the image to which parallax is assigned) may beseparated. Furthermore, depth information may be estimated withoutinputting the depth information (component thereof is not shown in FIG.3). In this case, if the depth information is estimated based on thediffuse reflection image, estimation accuracy thereof is improved.

Moreover, in following explanation, the depth information is representedas I^(depth). I^(depth) holds depth information of each pixel in theinput image I. Here, a depth corresponding to each pixel is representedas Z_(f). A range of Z_(f) is 0≦Z_(f)≦Z (For example, Z=1.0), 0represents the front side, and Z represents the back side.

Based on the depth information I^(depth) and the diffuse reflectionimage I^(diff) (by the separation unit 100), the generation unit 400generates a parallax image (S23). Specifically, a parallax vector d iscalculated from the depth information I^(depth), and each pixel of thediffuse reflection image I^(diff) based on the parallax vector d.Hereinafter, one example of a method for generating the parallax imagewill be explained in detail.

FIG. 5 is a schematic diagram to explain the method for calculating aparallax vector from the depth information. Here, b [cm] is a distancebetween both eyes, z_(s) [cm] is a distance to the screen, Z_(o) [cm] isa maximum projection distance from the screen to a foreground, and L_(z)[cm] is a depth distance in actual space. Parameters b, Z_(s), Z_(o) andL_(z) can be arbitrarily determined based on stereoscopic vision to bepresented. Here, the parallax vector d is calculated according to anequation (5).d=b{z′/(z _(s) +z′)}z′=L _(z) ·z _(f) /Z−z _(o)  (5)

The parallax image is generated by moving pixel values of the diffusereflection image according to the parallax vector d. An image for lefteye and an image for right eye (generated by moving the pixel values)are represented as I′^(diff) _(L) and I′^(diff) _(R) respectively. Whenthe parallax image is generated, as to a region to which pixel valuesare not assigned, the pixel values to assign to the region areinterpolated by other pixel values around the region.

In the same way as the first embodiment, the generation unit 200generates non-diffuse reflection image for left eye I′^(spec) _(L) andnon-diffuse reflection image for right eye I′^(spec) _(R) from thespecular reflection image I^(spec). In this case, by referring to thedepth information I^(depth), the generation unit 200 generates I′^(spec)_(L) and I′^(spec) _(R). For example, as shown in the equation (3),after the luminances are changed, in the same way as the generation unit400, a parallax is assigned to I′^(spec) _(L) and I′^(spec) _(R). As aresult, the stereoscopic vision can be reappeared while a difference ofthe luminance is set between I′^(spec) _(L) and I′^(spec) _(R).

The synthesis unit 300 synthesizes the diffuse reflection imageI′^(diff) _(L) with the specular reflection image I′^(spec) _(L), andgenerates the image for left eye I′_(L). In the same way, the synthesisunit 300 synthesizes the diffuse reflection image I′^(diff) _(R) withthe specular reflection image I′^(spec) _(R), and generates the imagefor right eye I′_(R) (S25). In the second embodiment, by assuming thedichromatic reflection model, in the same way as the equation (4), two(diffuse, specular) reflection images are synthesized according to anequation (6).I′ _(L) =I ^(diff) _(L) +I′ ^(spec) _(L)I′ _(R) =I ^(diff) _(R) +I′ ^(spec) _(R)  (6)

According to the second embodiment, from the input image and the depthinformation, a stereoscopic viewable image of which quality such asbrightness is improved can be generated.

(The Third Embodiment)

The third embodiment relates to an apparatus for generating outputimages corresponding to viewpoints different from the input image. Inthe third embodiment, an example that output images corresponding to atleast three viewpoints are generated from input images corresponding totwo viewpoints will be explained. Moreover, the third embodiment is notlimited to this example. The number of input images may be larger than(or equal to) two.

FIG. 6 is a block diagram of an image processing apparatus according tothe third embodiment. The image processing apparatus includes anestimation unit 201, a generation unit 202, and an estimation unit 500.The estimation unit 201 estimates/extracts a change of non-diffusereflection image among a plurality of input images. The generation unit202 generates non-diffuse reflection image corresponding to an arbitraryviewpoint by using the change of non-diffuse reflection image (estimatedby the estimation unit 201). The estimation unit 500 estimates a depthfrom at least two input images.

FIG. 7 is a flow chart of processing of the image processing apparatusaccording to the third embodiment. FIG. 8 is a schematic diagram toexplain all operation of the image processing apparatus in case that theinput image is (two) stereo images. Here, in the stereo images inputted,an image for left eye is I_(L), and an image for right eye is I_(R).Respective diffuse reflection images separated from the stereo images(by the separation unit 100) are I_(L) ^(diff) and I_(R) ^(diff).Furthermore, respective specular reflection images are I_(L) ^(spec) andI_(R) ^(spec).

The generation unit 200 estimates/extracts a change amount of the imageamong I_(L) ^(spec) and I_(R) ^(spec), and generates a specularreflection image corresponding to an arbitrary viewpoint based on thechange amount. Here, as shown in FIG. 9, viewpoints are horizontallyaligned at an equal interval. However, the viewpoints are not limited tothis alignment. In FIG. 9, the number of viewpoints is N, the viewpointsare C₀, . . . , C_(N−1), and images corresponding to the viewpoints areI₀, . . . , I_(N−1).

The estimation unit 500 estimates depth information I^(depth) from I_(L)and I_(R) (S32). In the third embodiment, the depth information isestimated by the stereo matching method. However, a method forestimating the depth information is not limited to this method.

The estimation unit 201 estimates a change amount between I_(L) ^(spec)and I_(R) ^(spec), and extracts the change amount to generate I^(spec)₀, . . . , I^(spec) _(N−1) (S33). Here, the change amount is representedas F. In the third embodiment, a correspondence relationship of pixelsbetween I_(L) ^(spec) and I_(R) ^(spec), and the correspondence vectorcan be extracted as F. Hereinafter, one example of a method forextracting F by using the correspondence relationship of pixels betweenI_(L) ^(spec) and I_(R) ^(spec) will be explained.

FIG. 10 shows one example of the method for extracting F by using thecorrespondence relationship between pixels. In this example, as tofollowing two aspects (1) and (2), the correspondence relationshipbetween two input images is thought about.

(1) Difference between two images due to a camera coordinate system(viewpoint position when stereo images are photographed):

(2) Difference between two images due to difference of reflected lightsdepending on a viewpoint position:

First, as to an image for right eye I_(R), by subjecting I_(R) to acamera coordinate transform to match the viewpoint position with animage for left eye I_(L), the acquired image D (I_(R)) is thought about.The camera coordinate transform D is calculated by using a parallaxvector (acquired from the depth information). In this case, as shown inFIG. 10, between I_(L) and D (I_(R)), the diffuse reflection imagesthereof coincide. However, the specular reflection images thereof do notcoincide. In this way, after the viewpoint position is matched betweenI_(R) and I_(L), a difference of images between I_(R) and I_(L) isextracted as the change amount F.

The difference of images between I_(R) and I_(L) depends on the specularreflection image only. Accordingly, by transforming I_(R) ^(spec) sothat the camera coordinate system is matched with I_(L) ^(spec), animage D (I_(R) ^(spec)) is generated. Next, by using I_(L) ^(spec) and D(I_(R) ^(spec)), the change amount F is estimated. Hereinafter, a methodfor estimating the change amount F will be explained.

By setting a difference of pixel values between I_(L) ^(spec) and D(I_(R) ^(spec)) to F, a change amount of luminance of gloss for eachpixel can be estimated.

Furthermore, by setting a vector field of a correspondence relationshipof pixels between I_(L) ^(spec) and D (I_(R) ^(spec)) to F, a shapechange amount of gloss for each viewpoint can be estimated. In order toestimate the correspondence relationship between two images, an opticalflow is used. As the method for estimating the correspondencerelationship, not only the optical flow but also any conventional methodmay be used.

The generation unit 202 generates specular reflection images I^(spec) ₀,. . . , I^(spec) _(N−1) of images I₀, . . . , I_(N−1) (S34). First, thegeneration unit 202 transforms the images I₀, . . . , I_(N−1) by usingthe change amount F. When a change amount of luminance of gloss is used,as shown in an equation (7), a multiple of constant of the change amountF is added.I ^(spec) _(n) =I _(L) ^(spec) +a*F  (7)

In the equation (7), if I^(spec) _(n) is smaller than (or equal to)zero, pixel values thereof is zero. Here, “a” is a coefficient of themultiple of constant. For example, by enlarging absolute value of “a”according to a distance between the viewpoint position and the centerposition, change of light intensity due to the viewpoint can berepresented.

Furthermore, when the shape change amount of gloss is used, as shown inupper part of FIG. 11, by moving each pixel value of I_(L) ^(spec) basedon the vector field of F, an image of which specular reflection image isenlarged/reduced/transformed is generated. In this case, by multiplyinga change amount with constant according to a distance between theviewpoint position and the center position, the specular reflectionimage gradually enlarging is represented. Above-mentioned two methods,i.e., a method by difference of luminance and a method by difference ofshape, may be used by combining.

A method for generating I^(spec) ₀, . . . , I^(spec) _(N−1) by multipleconstant of F is one example. As to F, the image may be generated bynon-linear extrapolation.

Next, by referring to the depth information, the generation unit 202assigns a parallax to I^(spec) ₀, . . . , I^(spec) _(N−1). In the sameway as operation of the generation unit 200 of the second embodiment, bycalculating a parallax vector from the depth information and by movingpixel values based on the parallax vector, the parallax can be assigned.

In the same way as the second embodiment, based on the depthinformation, the generation unit 400 generates parallax images I^(diff)₀, . . . , I^(diff) _(N−1) from the diffuse reflection image I_(L)^(diff) (S35).

The synthesis unit 300 synthesizes diffuse reflection images I^(diff) ₀,. . . , I^(diff) _(N−1) with specular reflection images I^(spec) ₀, . .. , I^(spec) _(N−1) respectively, and generates I₀, . . . , I_(N−1)(S36). In the third embodiment, by assuming the dichromatic reflectionmodel, in the same way as the equation (6), two (diffuse, specular)reflection images are synthesized.

According to the third embodiment, a stereoscopic image corresponding tomulti-viewpoint is generated from stereo images, and fell of thematerial (such as brightness) can be improved. In case of assuming thedichromatic reflection model, even if the specular reflection image ischanged by change of the viewpoint position, the pixel values are notsmaller than (or equal to) pixel values of the diffuse reflection image.However, if the diffuse reflection image is not estimated, it cannot bedecided whether the pixel values are smaller than (or equal to) pixelvalues of the diffuse reflection image. Accordingly, physical incorrectchange may be allowed.

On the other hand, in the third embodiment, a diffuse reflection colorof each pixel is separated from a specular reflection color of thepixel. Accordingly, above-mentioned failure does not occur. Furthermore,in the same way as the first and second embodiments, change executed tothe non-diffuse reflection image does not degrade the diffuse reflectionimage. Furthermore, when a difference of the non-diffuse reflectionimage is set between two images for right eye and left eye, the changeamount is estimated from the image. Accordingly, an image having morenatural quality can be generated.

While certain embodiments have been described, these embodiments havebeen presented by way of examples only, and are not intended to limitthe scope of the inventions. Indeed, the novel embodiments describedherein may be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An apparatus for generating a multi-viewpointimage from at least one target image, comprising: a processor configuredto: separate the target image into a first diffuse reflection image anda first non-diffuse reflection image based on a pixel value of eachpixel of the target image, the first non-diffuse reflection image havingcomponents except for the first diffuse reflection image; estimate achange amount of each pixel among a plurality of first non-diffusereflection images corresponding to viewpoints differently; generate asecond non-diffuse reflection image by changing at least one of a shapeand a luminance of each pixel of the first non-diffuse reflection image,based on the change amount of each pixel; and generate themulti-viewpoint image by synthesizing the first diffuse reflection imagewith the second non-diffuse reflection image, each viewpoint imagecorresponding to each of the viewpoints.
 2. The apparatus according toclaim 1, wherein the processor is configured to transform the pluralityof first non-diffuse reflection images into a coordinate system havingthe same viewpoint, and to estimate the change amount from acorrespondence relationship of each pixel among the plurality of firstnon-diffuse reflection images transformed.
 3. The apparatus according toclaim 1, wherein the processor is configured to generate a plurality offirst diffuse reflection images from pixel values of a plurality oftarget images corresponding to the viewpoints differently and toestimate the change amount from a correspondence relationship of eachpixel among the plurality of first diffuse reflection images.
 4. Theapparatus according to claim 1, wherein the processor is configured toestimate a depth information from the first diffuse reflection image ofthe target image.
 5. The apparatus according to claim 1, wherein themulti-viewpoint image includes the target image.
 6. The apparatusaccording to claim 1, wherein the processor is configured to generatethe multi-viewpoint image by synthesizing the first diffuse reflectionimage with the plurality of first non-diffuse reflection images havingpixels of which at least one of the shape and the luminance is changedbased on the change amount of each pixel.
 7. The apparatus according toclaim 1, wherein the processor is configured to transform the signalvalue of the first non-diffuse reflection image by a plurality of gammavalues respectively, and the transformed signal value is used forgenerating a synthesized image to be presented to the viewpointsrespectively.
 8. The apparatus according to claim 1, wherein theprocessor is configured to detect a region of which the signal value ofthe first diffuse reflection image is larger than a specific threshold,and to change at least one of a size and a position of the region.
 9. Amethod for generating a multi-viewpoint image from at least one targetimage, comprising: separating by a processor, the target image into afirst diffuse reflection image and a first non-diffuse reflection imagebased on a pixel value of each pixel of the target image, the firstnon-diffuse reflection image having components except for the firstdiffuse reflection image; estimating by the processor, a change amountof each pixel among a plurality of first non-diffuse reflection imagescorresponding to viewpoints differently; generating by the processor, asecond non-diffuse reflection image by changing at least one of a shapeand a luminance of each pixel of the first non-diffuse reflection image,based on the change amount of each pixel; and generating by theprocessor, the multi-viewpoint image by synthesizing the first diffusereflection image with the second non-diffuse reflection image, eachviewpoint image corresponding to each of the viewpoints.